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1.  The  reports  listed  on  Inclosure  1  are  approved  for  public  release  with  unlimited 
distribution  (95  numbered  ARI  Research  Memorandums,  51-1  thru  52-82). 

2.  These  are  among  the  unrecorded  ARI  reports  which  you  identified  to  us  22  June 
1979  as  not  in  your  retrieval-  system.  The  accompanying  box  contains  at  least  one 
copy  of  each  report  listed  in  Inclosure  1,  for  your  retention  and  reproduction. 


3.  We  will  be  sending  more  previously  unrecorded  ARI  reports  to  you  shortly. 
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Id  oonoectlon  vith  th«  Career  Guidaoc®  Program  of  th®  Ars^,  to  onlietcd 
efficionoy  report  (BER)  tram  defeloped  to  oorer  thoao  aapects  of  pex'forjstooe  of 
enlisted  laen  which  wore  not  to  bo  aeasiarod  bj  the  proficiency  teats.  The  SKR 
put  Into  official  uao  in  September  1948  was  the  outgrowth  of  two  esperlaentml 


forma,  the  first  of  which  1#  described  In  the  proeent  report.^ 


\ 


Tho  first  experimental  form  utilized  three  types  of  items:  (1)  20  Items 
(l»soi*ibing  personality  oharaoterietios,  each  persaittlng  a  rating  of  the  enlisted 
ttiaa  (Hi)  on  a  throe-point  scale  as  "lowest  "middle  or"hlgho8t  of  all 
man  of  equal  grade  and  respocslbility;  (2)  selection  of  fire  of  these  20  Itema 
as  "aost  descriptive"  and  fits  a#  "least  desorlptlve"  of  the  SM,  a  modlfloatlon 
of  the  forced  choice  technique;  sod  (J)  estimated  psrfonssnoe  of  the  XK  at  «aoh 
of  the  five  upper  enlisted  pty  grsdes  as  a  teohnlced  epeolaliat  sjod,  as  a  aapsr- 
vieor  or  an  admlnlstratlTe  worker,  by  ae&ns  of  five-point  rating  s<^ss  with 
scale  points  described.  The  primary  pui^se  of  this  first  form  was  to  explore 
the  possibilities  of  these  ssveral  types  of  items. 


Tho  XXS  was  administered  along  with  two  criterion  instnoieotfl,  evaloationa 
on  oTsr-all  Job  perforaanoe;  one  was  a  20-polnt  rating  scale,  ax»d  tshe  otfeor  a 
noailnatlng  form  on  which  the  raters  iodlcatod  the  firs  most  and  fire  Least 
sffeotiw®  asT).  The  three  foras  were  completed  for  970  enlisted  men  repressnta- 
tirs  of  all-ndlltary  occupational  specialties  (iCS’s)  and  of  the  v^per  flT®  pay 
grades.  In  addition,  raters  and  indorsers  filled  out  a  questionnaire  about  this 
DGR  to  suggest  fuz^tber  dawelopcunt  and  utilization  of  efficiency  reports,  fhs 
oorrslation  between  soosres  00  the  two  criterion  Instruaenta  wa*  found  to  be  .71 
fsa*  a  sample  ooj^pos&d  of  oases  with  four  or  more  raters.  Vsxious  astbods  of 
scoring  the  experimental  BR  were  tried.  The  indications  were  that  tho  fom 
could  b®  scored  to  yield  a  ralldlty  of  about  .40  for  the  fire  grades  ocaablneHi. 
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Regardleoa  of  method  of  eoorlng,  the  report  yielded  the  lo^foat  validity  for 
grade  H  men  (at  best  around  .25) .  Thie  lover  validity  was  a  serious  defect^ 
since  the  stop  from  grade  1*  to  the  next  higher  grade  represente  one  of  tlie  more 
orltlcal  aeloctlcD  points  in  the  Career  Guidance  Program.  (Supervisory  or 
leadership  ability  measured  by  the  ESP,  is  required  In  the  higher  gradsa  as  well 
aa  tecbnlced  competence  or  skill,  measured  by  the  proficiency  teste.)  Considered 
as  single  Iteaa,  the  modified  forced  choice  technique  yielded  the  lowest  Talldity, 
(most  r*8  below  .20);  the  three-point  rating  scales  wore  next,  (moot  r's  between 
.20  and  .50);  end  the  fivo-point  over-all  typo  rating  scales  highest,  (jaost  r'a 
above  .50). 

The  typos  of  items  used  had  two  limitations.  The  first  concerns  the  usual 
finding  for  responaos  to  graphic  items.  People  do  not  like  to  ropcjrt  unfavorable 
tilings  when  rating.  On  the  average  only  6^  of  the  group  was  rated  aa  being  in 
the  "lowest  y  categoirjr  of  the  throe-point  scales.  The  second  limitation  concerns 
the  modified  forced  choice  technique.  The  numbor  of  tinies  an  item  was  chechsd  ea 
most  or  least  descriptive  of  ratoea  was  found  to  be  related  to  its  position  la  the 
list.  Itows  near  the  beginning  vers  checked  most  frequently. 

On  tlio  basis  of  these  findings,  it  was  recoanended  that  different  forced 
choice  techniques  bo  tried  sod  that  further  experimentation  be  conducted  with 
rating  scales  before  items  wore  selected  for  an  official  enlisted  effloienoy 
report. 


FR0BL15M 

Pspor  and  pencil  type  proficiency  testa  end  Bnllsted  Kfflolency  Heporta 
(22E)  are  developed  by  the  Personnel  Research  Section,  AGO,  as  part  of  the 
proaotlOTj  procedures  of  the  Career  Guidance  Plan  for  ©nllated  personnel. 


D/F  from  d/p  and  A  to  TAG,  File  210. 5I  (6  Aug  *i6) ,  Gubject;  "Career  Guidance  E 
for  Personnel  Belov  Conalssloned  Gratia,"  dated  26  June  19^7.  3o«  also,  U 

PR-UllO,  Program  Plan  for  Developswnt  end  Validation  of  an  Sffloiency  Report  L 

for  Personnel  Below  Ooiatiaslonod  Grade,  dated  15  July  19^7 •  - - 

Cir  1,  Career  Guidance  Plan  for  Varrant  Officers  and  JEnlisted  Personpel, 

Department  of  the  Army,  1  January  19^(B.~  - - - 

_Difitributlon/ _ _ 

Availability  Codas 

_  2  -  Availand/oi- 

Dist  special 


The  SRB  »a«  deaignod  to  ooT*r  aepacte  of  Job  porformauce  ouch  aa  fluparrleorj 
said  l*«dersiilp  ability  not  raoMurod  by  th®  proflclenoy  exealnationa .  SeYoral 
other  report*  of  »ieR  reeaarcli  are  being  propca>&d  for  publication.^  The  official 
XSa  (M  AGO  yorm  297)^  wa*  Introduced  vlth  the  I’oud  Sorfloe  Caraer  Field  in 
October  19^8,^  «od  ww  u*«d  vlth  other  career  field*  as  they  vero  instituted. 

The  requireMBat*  to  which  the  SSSR  bed  to  oonfona,  lljaited  to  80fl»  extent 
it*  fona  and  content.  The  Job*  held  by  enlisted  oen  differ  «o  widely  that  it 
waa  not  feasible  to  use  Itea*  which  were  too  specific  to  «ay  one  Job  or  type  of 
Job.  The  ESR  had  to  be  eaaily  under*tood  and  not  too  lengthy^  and  the  score 
had  to  be  on©  that  could  be  exproased  by  a  elngla  number. 

This  attsly,  th©  initial  phase  in  the  deTelopoaent  of  an  enlisted  effiolenoy 
report,  was  designed  to  InTeatlgate  •eTeral  types  of  rating  devlcea.  Included 
wore  three-point  and  fiTe-polnt  scales,  and  a  acdlfled  forced  choice  technique. 

KSTHOD 

Population 

The  first  experioental  Knllsted  Bfficlonoy  Report  (Printing  Job  Ifo.  Vf  iH&9) 
was  ooinplotod  for  970  men:  at  3oott  Field,  and  yih  at  Fort  Boox,  ia  Asgust 

19**’T  (3*«  Table  1,  page  U) .  The  zaan  rated  were  reparesontatlt©  of  *illt*ary 
oooypatlonal  •peolaltles  ()t)8’s)  at  these  Inetallatloos  and  of  the  upper  fire 
enlisted  pay  grades. 


^  I®a  Report  8Mi,  Dere^pwant  of  the  Bnlleted  IKffloieooy  Beport;  II  The  Seoowd 
axperiaental  Fora  *ol’  ^  Offlctkl  Fora. 

leport  8h7,  Analjela  cf  the  Snlisted  Iffloteocy  Beport  In  the  Operatiaa 
yrograa. 

PH3  Report  655,  DeTelopwent  of  a  Mew  Siyetea  for  Soorlog  the  ii^llsted 
affiolenoy  Report. 

^  AH  600«l80,  ParsQonel:  Hnllated  Fffiolenoy  Reports,  Departawnt  of  the  Ariay, 
1?  3ept«aber~I5S®^  !  ”” 

^  Glr  205,  Iotrodi>otloo  of  Food  3emo»  Career  Field,  Dapartasent  of  th©  Anay, 

8  July  19511 
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POPOL/^TION  FOB  3!?UI)Y  Of  PIHST  KII^IMJCNTAL  KlLIST'ffl  EfP  rCtSNCY  RSPOKT 

BT  ORAI®  AiiB  ESarAIJoATIOK 


Grade 

Title 

New 

Title# 

Hew 

Grade*-* 

Fort 

&OX 

.Scott 

Field 

Total 

1. 

Master  sei-geant 

Master  sergeant 

l!-7 

65 

55 

120 

First  aorgoant 

2 

Technical  sergsafit 

Sergeant,  first  class 

K-6 

70 

35 

105 

3 

Staff  sergeant 

Sergeant 

K-5 

111 

108 

219 

k 

Sergeant 

Corporal 

N-4 

ll7 

155 

280 

3 

Corporal 

PrlTate,  first  class 

3-3 

151 

115 

2h6 

Total 

■'52V 

■*  Kffactivo  1  August  Clr  202,  iJEq^Ieaant&tlon  of  Ceroor  GrUldaoco  Pita  for 

Varrant  Officers  and  £nliated  Pargonngl,  IBspartiaont  of  th«  ArayT^T'^wij'  19^' 


Joint  Array  and  Air  Torae  Bulla  tin  No.  JO,  21  October  1S'^9>  Career  Coapengatlon 
Act. 


Batorc  and  rate©*  vare  preselected.  The  three  rating  fcwraia  and  a  questioonair# 

(See  ^InotnuBaDts"  belov)  were  distributed  at  the  inetallatlons  under  coTsr  of 
a  fort*  letter  vhloh  briefly  explained  the  purpose  of  the  sttBiy  end  the  routing 
of  awiterial#  froa  rat<5r  to  indorser  to  collection  point. 

iKgremKarrs 

1.  rhe  ifnlisted  Sfflclenoy  Beport  (Printing  Job  No.  kj  Ih6l9,  no  PHSf  nuzabei')  • 
The  first  group  of  Itenw  contained  tifenty  pteasofl  describing  porsoBallty  obarsoter- 
1 sties  oo  ifkich  the  2M  was  to  be  rated,  for  each  ptoase  the  rater  •tfs*  first  astoed 
to  indicate  on  a  three-point  scale  whether  the  BK  bolooged  to  tlK»  lowest,  adddlo, 
or  Mghset  thiird  of  all  »en  of  equal  grade  and  lasponelblllty.  He  was  tlseo  «ab»d 
to  inditsato  TSfhlch  fife  of  the  twenty  phraces  were  aost  desoriptlre  of  tJie  tma 
being  rated,  and  which  fire  were  le&et  daecrlptlT©,  a  aodlfloatloo  of  the  forced 
choice  teoimlque.  Finally,  tho  rater  va*  instructed  to  decide  without  regard  to 
precent  grade,  how  well  the  rates  would  poi’fora  at  each  of  the  fife  highest  non- 
ooBSBiiBsiooed  grades  in  his  (the  rates' b)  area  of  technical  epeoiailxatlon,  arjd 
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in  an  adminiatmtiTo  ov  .cuparrlsorj  capacity.  luforraatlon  identifying  the 
rate®  and  autbsntlcatlon  by  jratar  ai)d  by  indox'ser  wore  Included.  Only  one 
rating  form  vaxs  completed  for  each  ratee,  and  the  rater  In  each  case  vaa  the 
laB»dlat»  superrlior  of  th*  ratoo. 

2.  Criterion  la*tru»9nt«.  Two  aeeoclate  rating  foi'M  vsro  dereloped  for 
obtaining  criterion  evaluatlone  aa  a  baeU  for  Talidatloo  of  the  firet  experi.- 
taontal  KER.  Boatera  vere  mlaioographed  on  both  forms.  !Sfeoh  rater  vae  instmoted 
to  ♦lisd.nato  from  oonBldoratlon  hie  own  naoie  and  the  name  of  any  man  whom  ha  did 
not  Itnow. 

a.  The  Bnliated  Man  Kvaluation  Form  EXK-la/  iff)  AGO  HIT  767/  prorided 
for  evaluation  of  over-all  Job  performance  on  a  rating  tical©  with  twenty  dlYlaions 
or  boxes  rar)e;ing  from  1,  "Poorest,”  to  20,  "Best."  The  laen  on  the  roster  wera 
ocMSpared  with  a  represontativ©  group  of  the  same  gi’ad#  and  general  responBlbilitiea. 
The  criterion  aoore  on  the  EME-la  waa  the  ntaaerlcal  rains  of  the  box  ou  the  rating 
scale  Id  which  the  ratee* a  roster  number  waa  placed. 

b.  The  Xnliated  >kui  Sraluatlon  Fora  JBW-lb/  WO  AGO  Ha'  768,  required 
an  eraluatlon  of  orer-aLl  Job  performeaoe  without  regard  to  grade  by  obooeiog 
the  five  "meat  affective  men"  and  then  five  "least  effectir©  men"  from  thoae  on 
the  roster  Iznown  to  the  rater.  Criterion  sooraa  for  the  BWS-lb  were  obtained  M 
ahown  in  Table  2,  page  6. 

3-  QuoatioDnalre  on  Snllated  Ifficlenoy  Report,  HU  ACO  PRI  766  was  given 
at  the  time  of  the  admlniatratioo  of  the  first  eiperiaantal  for*.  Eeoulta  of 

‘  ^  5  ^  1  ^ 

tlila  study  are  analytod  In  EBfi  Report  765  ‘  i 

HiCXrKWJHlB  AHU  BSSULTS 


The  oori'siatiooa  between  acoree  on  the  two  oriterloa  Inrtjrusajnts  aaCS-1* 
and  lb  were  found  to  bo  .71  for  th©  coabinod  populations  (.75  for  &  sample  of 
335  oases  at  Scott  Field,  and  .JO  for  427  oaoes  at  Fort  Knox).  The  samples 
were  conxpoaed  of  cases  with  four  or  more  raters. 


PR3  Report  765,  Analysis  of  Results  of  Queotlonnairo,  PET  766,  00  SnllBtod 
JKfficlency  Beport/  I6  August  1948. 


TABLE  2 

wBrcsra  used  m  obtaihikg  CRrmiioN  scores 

FOR  KaLISTKD  MAN  EVALUATION,  EMS-lb 


No.  of  Ken 
Considered 


Wol^t  of  Rating 

Moet  EffectiTo  Men  Leaiat  Effectir*  Men 


Beat  125^ 


Pooreet 


l4  or  Eoro 

11-15 

8-10 

5-7 

3-4 


5  2  11 
3211 

5  2  I 

2  1 
1 


-1  -1  -2  -3 

-1  -1  -2  -5 

-1  -2  -3 
-1  -2 
-1 


Dlrectlone  for  using  Chart 

Nuaber  of  .uen  considered  is  the  number  of  names  on  the  roster  not  crossed 
out.  Use  weights  assigned  for  "Most  Effect! to  Men"  and  "Least  KffectlTe  Hen" 
regardless  of  position  on  rating  scale  In  which  a  man  may  hare  been  pla-cad.  for 
eiamplo,  If  only  fire  men  are  rated  and  entiles  are  made  in  positions  1  throu^  5 
under  "most  effect! to,"  position  5  vlli  be  weighted  -2  and  position  4  weighted  -1. 

The  Enlieted  Sfflclenoy  Yalldi'tj^eport^ 

In  Table  5  are  given  the  validltlea  by  grade  of  the  five-point  rating 
scales  on  eetlaated  porformanco  in  Technical  Spooialisetlon  and  in  AdrainlstnAtion 
and  Superrision  at  the  upper  fire  enlieted  pay  grades.  The  most  clear-cut  find¬ 
ing  was  the  tendency  for  the  ratings  to  be  more  valid  for  the  higher  current  grades. 
One  explanation  aay  be  that  iodivldual  differences  beoomo  more  apparent  under  the 
greater  Job  duty  requlroaants  of  the  higher  grades,  llie  validity  for  grade  4  men 
V8«  particularly  disappointing.  Because  the  step  from  grade  4  to  the  next  higher 
grade  is  an  important  one  involving  Buperrisory  or  leadership  ability,  an  enlletad 
efficiency  report  must  be  able  to  differentiate  among  grade  4  men  if  it  is  to  be 
of  real  usefulness. 

There  were  no  clear-cut  tendencies  (Table  3)  for  estimatea  of  performance 
in  TeohnlosI  Specialization  to  differ  in  validity  from  those  in  Adainiatr&tion  or 
Supervision  or  for  ratings  on  present  grade  to  differ  from  ratings  in  next  higher 
grade  or  in  the  highest  grade. 
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Tabl«  contAlne  tho  Itom  Talidltlen  of  the  20  phraa##  for  to©  thre©- 
polot  rating  scale©  aod  for  the  nodlfied  foi*ced  ohoio«  t*o]joiq,ue.  The  raliditlea 
tor  tba  three -point  aoale*  were  In  general  higher  (rsage  .07  to  .hl^  with  aoet 
QboT«  .20)  than  for  the  modified  forced  choice  technique  (range  ,00  to  .38,  with 


$ 

.i- 


il 

*:5i 


aoBt  bolov  .20),  but  those  for  the  fire-point  eoalea  (Table  3)  wre  highest 
(range  .02  to  .52  with  jncat  abore  .30). 

TA3LS  3 

i. 

VALIOiriES*^  OF  FITE-POINT  3CAI.2S,  FIRST  SXP3EIMg!Fim  XHLiaCH)  ETFIOISdCY  | 

REPORT  3T  EMLiarSD  GRADE  AGAIRST  CRIT3ERI0R  IHSTHIM3fflTS  BMB-U  ASD  2MS-lb 


S  «  87C' 


Technical  Speclall zation 

Administration  or 

Superrlaion 

Criterion 

Grade** 

H 

Rating  Estimated  Perfomianoo 
in  iloxt  Gra«de 

Current  Higher  1 

Grade  Grade 

Rating  Estimated  Performance 
in  holt  Grade 

Current  Higher  1 

Grade  Grade 

1 

120 

.52 

.52 

.42 

.42 

.2 

105 

.46 

.43 

.46 

.30 

.32 

.53 

EME-1« 

3 

219 

,40 

.38 

.45 

.32 

.55 

.45 

U 

180 

.02 

.10 

.05 

.05 

.24 

.19 

'i 

✓ 

2b6 

.26 

.27 

.29 

.10 

.25 

.21 

Total  R 

070 

At  .r  ,54 

.30 

.56 

,26 

■  .53 

.33 

1 

120 

.50 

.46 

2 

105 

.55 

.59 

.25 

.31 

EMS-lb 

5 

219 

.47 

.46 

.56 

.44 

h 

ISO 

-.01 

.08 

.04 

.12 

c. 

246 

.28 

.50 

.13 

.26 

Total 

870 

At  .r  ,33 

.54 

.25 

.28 

♦BiaorlAl.  Item*  wer©  dichotomlted  to  obtain  ae  nearly  a  50"50  aplit 
as  possible. 


**Ses  Table  1. 


It  auouifli  be  noted  tha-  liohot«ml2lng  the  Item  between  the  "highest  ■=:" 
and  ”ald<31e  catetVJriea  was  about  equivalent  to  di oho tcrd zing  at  the  »lddle 
of  the  dlstrlbutlop ,  for  the  uBual  dlslncllnetion  of  raters  to  use  the  low  end 
of  the  acale  (i’eport  iinfeTorabl7  about  the  retoe)  was  obaerred.  The  distribu¬ 
tion  of  reaponaes,  baaed  on  an  averege  for  all  20  phraaee  was;  "highest 

59^  "middle  4,"  aJid  6^  "lowest  J." 

A  finding  that  obscured  the  interpretation  of  the  validity  coefficient  for 
the  modified  forced  choice  technique  was  the  strong  tendency  for  raters  to  use 
the  descriptions  at  the  beginning  of  the  list  (lore  frequently  than  those  at  the 
end  (See  Fig.  1) . 

In  addition  to  obtaining  the  validities  of  the  first  20  iteae  (Table  4), 
the  validities  of  total  ocore  ors  these  Items,  using  different  scoring  methods, 
were  estimated  with  EMS-la  as  a  criterion.  Instead  of  scoring  the  items  on  a 
throe-point  scale,  the  following  three  methods  were  used: 

a.  Counting  the  number  of  "highest  responses 

b.  Counting  the  number  of  "lowest  ^  responses 

•  c.  Combining  methods  "a"  and  "b"  with  multiple  regression  weight* 

The  results  are  given  in  Table  5*  Th*  lower  means  and  standard  deviations 
for  thoT-cr/eat.  4"  scoring  reflect  the  tendency  for  raters  to  avoid  the  lowest 
scale  ui2lte.  In  general  the  "highest  ^  method  yielded  validities  superior  to 
the  "lowest  ^  jwsthod.  Combining  the  two  with  multiple  regression  wel^ts  yielded 
the  highest  validities  of  all.  Those  results  have  two  implications;  (1)  the 
method  of  scoring  favorable  and  unfavorable  positions  separately  and  then  oombia- 
ing  to  obtain  a  total  acore,  appears  to  have  promise;  and  (2)  an  over-all  validltl' 
of  around  .40  could  be  obtained  in  this  way*  It  should  be  noted  that  validities 
were  again  lowest  for  grade  4  (.27,  .25).  However,  with  this  method  of  scoring, 
the  validity  at  grade  4  was  sostowhat  better  than  for  the  5«point  scales,  see 
Table  5  ( .01  to  .05  for  rating  In  current  grades),  and  the  differenoes  in 
validity  for  the  vsarlous  enlisted  grades  was  not  as  pronounced. 
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In  T*b3j>  6  are  pro»@nted  the  intercorrolatlons  of  tJa©  tlxree -point  eoal«B. 

Tfeo  interoorrclation*  rumgdd  from  .12  to  .65.  In  general  thej  wore  higher  than 
the  5-point  aottle  itsta  valtdltlee  (Table  5) .  They  are  Higher  than  la  deairad 
from  tha  atandpoint  of  combining  all  items  for  affective  prediction. 

RiSC0l*C0tDAJT0H3 

The  tendancj  of  the  raters  to  check  taoro  frequently  those  iteos  iqppearlng 
at  the  beginning  of  t)ie  modified  forced  choloe  Hat  should  be  studied  to  determine 
vhether  the  Hat  vae  too  long  to  be  kept  in  alnd  by  the  ratera,  or  vhether  the 
leng:th  of  the  teak  lowered  owtlvation. 

On  the  basis  of  ths  findings  with  the  form>  the  daclaion  was  made  to  develop 
a  second  experimental  fona  embodying  different  types  of  items.  Tljs  20  graphic 
items  frot  the  first  form  were  to  be  Incorporated  Into  forced  choice  paire. 

Turthor  research  is  rocoicaendod  on  the  problem  of  grade  bias,  since 
considerable  dlfferonooe  according  to  grade  were  reroalod  in  this  study. 
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